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WHAT IS CLAIMED IS : 

1. A method for speech recognition, comprising the 
steps of: 

generating a set of likely hypotheses in recognizing 
5 speech; 

rescoring the likely hypotheses by using semantic 
content by employing semantic structured language models; 
and 

scoring parse trees to identify a best sentence 
10 according to the sentences ' parse tree by employing the 

semantic structured language models to clarify the 
recognized speech. 



2. The method as recited in claim 1, further 
15 comprising the step of training a language model using 

speech recognition methods. 



3. The method as recited in claim 1, wherein the set 
of likely hypotheses is in the form of an N-best list or 

20 lattice. 

4. The method as recited in claim 1, wherein the 
step of rescoring employs MELM2 or MELM3 semantic 
structured language models. 
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5. The method as recited in claim 1, wherein the 
step of scoring parse trees to identify a best sentence 
according to the sentence's parse tree by employing the 
5 semantic structured language models to clarify the 

recognized speech includes the step of training the 
structured semantic language models in accordance with 
history parameters and history questions. 

10 6. The method as recited in claim 5, wherein the 

history parameters include a previous word (wj-1), a 
previous word of the previous word (wj-2), a parent 
constituent label (L) , a number of tokens (N) to the left 
since L starts, a previous closed constituent label (0) , a 

15 number of tokens (M) to the left after O finishes, and a 

grandparent label (G) . 

7. The method as recited in claim 5, wherein the 
history questions include a default, (wj-l) , (wj-l, wj-2) , 
20 (L,N), (0,M), and (L,G) . 
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8. The method as recited in claim 1, further 
comprising the step of determining a confidence 
measurement . 

5 9. The method as recited in claim 8, wherein the 

step of determining a confidence measurement includes 
employing a statistical method to combine word sequences 
with a parser tree to determine a confidence score for 
recognized speech. 

10 

10. The method as recited in claim 8, wherein the 
step of determining a confidence measurement includes 
employing scores obtained from the semantic structured 
language models along with other speech recognition based 

15 features. 

11. The method as recited in claim 1, further 
comprising the step of extracting probabilities assigned to 
tags, labels and extensions obtained from a parser tree. 

20 

12. The method as recited in claim 11, further 
comprising the step of combining the semantic structured 
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language models and speech recognition based features with 
the extracted probabilities using a classifier. 

13. The method as recited in claim 1, wherein the 
5 semantic structured language models are trained by 

employing unigram, bigram and trigram features. 

14. The method as recited in claim 1, wherein the 
semantic structured language models are trained using one 

10 or more of relative labels, token numbers, and answers to 

questions related to word order or placement. 

15. The method as recited in claim 1, wherein the 
semantic structured language models are trained by 

15 including a unigram feature, a bigram feature, a trigram 

feature, a current active parent label (Li) , a number of 
tokens (Ni) to the left since current parent label (Li) 
starts, a previous closed constituent label (Oi) , a number 
of tokens (Mi) to the left after the previous closed 

20 constituent label finishes, and a number of questions to 

classify parser tree entries. 
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16. The method as recited in claim 15, wherein the 
questions include a default, (wj-1) , (wj-l, wj-2) , (Li), 
(Li, Ni) , (Li, Ni, wj-l), and (Oi, Mi), where w represents 
a word and j is and index representing word position. 

5 

17. A program storage device readable by machine, 
tangibly embodying a program of instructions executable by 
the machine to perform method steps for speech recognition, 
in accordance with claim 1. 

10 

18. A method for speech recognition, comprising the 
steps of: 

generating a set of likely hypotheses in recognizing 
speech; 

15 rescoring the likely hypotheses by using semantic 

content by employing semantic structured language models; 
and 

scoring parse trees to identify a best sentence 
according to the sentence's parse tree by employing the 
20 semantic structured language models to clarify the 

recognized speech; and 



-28- 



Express Mail # EV314133019US 



Attorney Docket: YOR920030404US1 (163-13) 



determining a confidence measurement by employing 
scores obtained from the semantic structured language 
models along with other speech recognition based features. 

5 19. The method as recited in claim 18, wherein the 

set of likely hypotheses is in the form of an N-best list 
or lattice. 

20. The method as recited in claim 18, wherein the 
10 step of rescoring employs MELM2 or MELM3 semantic 

structured language models. 

21. The method as recited in claim 18, wherein the 
step of scoring parse trees to identify a best sentence 

15 according to the sentence's parse tree by employing the 

semantic structured language models to clarify the 
recognized speech includes the step of training the 
semantic structured language models in accordance with 
history parameters and history questions. 

20 

22. The method as recited in claim 21, wherein the 
history parameters include a previous word (wj-1) , a 
previous word of the previous word (wj-2), a parent 

-29- 



Express Mail # EV314133019US 



Attorney Docket: YOR920030404US1 (163-13) 



constituent label (L) , a niimber of tokens (N) to the left 
since L starts, a previous closed constituent label (0) , a 
number of tokens (M) to the left after O finishes, and a 
grandparent label (G) . 

5 

23. The method as recited in claim 21, wherein the 
history questions include a default, (wj-1) , (wj-1, wj-2), 
(L,N) , (0,M) , and (L,G) . 

10 24. The method as recited in claim 18, further 

comprising the step of extracting probabilities assigned to 
tags, labels and extensions obtained from a parser tree. 

25. The method as recited in claim 24, further 
15 comprising the step of combining the semantic structured 

language models and speech recognition based features with 
the extracted probabilities using a classifier. 

26. The method as recited in claim 18, wherein the 
20 semantic structured language models are trained by 

employing unigram, bigram and trigram features. 
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27. The method as recited in claim 18, wherein the 
semantic structured language models are trained using one 
or more of relative labels, token numbers, and answers to 
questions related to word order or placement. 

5 

28. The method as recited in claim 18, wherein the 
semantic structured language models are trained by 
including a unigram feature, a bigram feature, a trigram 
feature, a current active parent label (Li), a number of 

10 tokens (Ni) to the left since current parent label (Li) 

starts, a previous closed constituent label (Oi) , a number 
of tokens (Mi) to the left after the previous closed 
constituent label finishes, and a nuinber of questions to 
classify parser tree entries. 

15 

29. The method as recited in claim 28, wherein the 
questions include a default, (wj-1) , (wj-1, wj-2), (Li), 
(Li, Ni), (Li, Ni, wj-1), and (Oi, Mi), where w represents 
a word and j is and index representing word position. 

20 

30. A program storage device readable by machine, 
tangibly embodying a program of instructions executable by 
the machine to perform method steps for speech recognition, 
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in accordance with claim 18. 

31. A system for speech recognition, comprising: 

a unified language model including a semantic language 
5 model and a lexical language model; 

a recognition engine which finds a parse tree to 
analyze a word group using the lexical model and the 
semantic models, wherein the parse tree is selected based 
on lexical information and semantic information which 
10 considers tags, labels, and extensions to recognize speech. 

32. The system as recited in claim 31, wherein the 
parser tree includes semantic information and classer 
information used in identifying a best parser tree for a 

15 given word group. 

33. The system as recited in claim 31, wherein the 
parser tree includes information extracted from parsed 
sentences to statistically model semantic and lexical 

20 content of sentences. 
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34. The system as recited in claim 31, wherein the 
semantic language model includes unigram, bigram and 
trigram features. 

5 35. The system as recited in claim 31, wherein the 

semantic language model includes one or more of relative 
labels, token numbers, and answers to questions related to 
word order or placement . 

10 36. The system as recited in claim 31, wherein the 

semantic model is trained by including a unigram feature, a 
bigram feature, a trigram feature, a current active parent 
label (Li) , a number of tokens (Ni) to the left since 
current parent label (Li) starts, a previous closed 

15 constituent label (Oi), a number of tokens (Mi) to the left 

after the previous closed constituent label finishes, and a 
number of questions to classify parse tree entries. 

37. The system as recited in claim 36, wherein the 
20 questions include a default, (wj-1) , (wj-1, wj-2) , (Li), 

(Li, Ni) , (Li, Ni, wj-1), and (Oi, Mi), where w represents 
a word and j is and index representing word position. 
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38. The system as recited in claim 31, wherein the 
semantic model is trained by including history parameters 
and history questions, 

5 39. The system as recited in claim 38 wherein the 

history parameters include a previous word (wj-1), a 
previous word of the previous word (wj-2), a parent 
constituent label (L) , a number of tokens (N) to the left 
since L starts, a previous closed constituent label (O) , a 
10 number of tokens (M) to the left after 0 finishes, and a 
grandparent label (G) . 

40. The system as recited in claim 39, wherein the 
histoiry questions include a default, (wj-1), (wj-1, wj-2), 

15 (L,N), (0,M), and {L,G) . 

41. The system as recited in claim 31, further 
comprising a confidence measurement module. 

20 42. The system as recited in claim 31, wherein the 

confidence measurement module employs a statistical method 
to combine word sequences with the parse tree to determine 
a confidence score for recognized speech. 
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43. The system as recited in claim 31, wherein the 
confidence measurement module extracts probabilities 
assigned to tag nodes, label nodes and extensions in the ^ 
5 parse tree. 



-35- 



